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|  ABSTRACT 

There  is  a  one-to-one  relationship  between  a  set  of  P  normalized  positive  definite  correlation  esti¬ 
mates  and  the  P  predictor  coefficients  derived  using  autoregressive  modeling.  Several  renarchers  have 
proposed  the  use  of  M  >  P  oorrelatian  to  provide  a  better  Pth  order  model.  SpedficaDy,  the 

"—■mat  equations  are  augmented  to  provide  M  linear  equations  between  the  correlation  estimates  and 
the  predictor  coefficient!.  Since  the  system  of  equations  is  now  ove: specified,  a  least  squares  solution 
is  required. 

la  this  thesis  a  study  is  presented  of  some  of  the  properties  of  the  method  of  overspecified  nor¬ 
mal  equations  as  applied  to  d tie  problem  of  spectral  emimatiaa.  The  main  contribution  of  this  thesis  is 
the  derivation  of  die  relationships  between  the  number  of  correlations  used,  the  model  order  and  the 
rignal  to  noise  ratio  of  die  rignal,  to  die  characteristics  of  the  resulting  spectral  estimate.  The  charac¬ 
teristics  studied  are  the  spectral  height,  bandwidth  and  area.  The  method  is  shown  to  be  a  spectral  den¬ 
ary  estimator  She  the  ME  method,  where  spectral  anas  rather  than  spectral  values  should  be  inter¬ 
preted  as  eaimattn  of  power. 

Hie  relationships  derived  point  to  the  *"""***>•  of  correlations  used  over  the  minimum,  i.e.  model 
order,  as  an  signal- to-noiae  enhancer.  The  resulting  «p-*"»m  is  equivalent  to  the  ME  spectrum  under 
higher  rignal- to-noiae  conditions.  Another  result  is  the  requirement  of  a  proportionality  constant 
dependent  an  the  number  of  correlations  and  the  model  order  which  is  necessary  for  unbiased  rignal- 
I  to-nosse  measurements^  This  constant  is  not  required  however,  for  measurements  of  relative  power 
within  the  same  spectral  estimate,  as  in  the  power  ratio  of  two  sinusoids  in  noise. 

The  second  part  of  the  tberis  presents  some  empirical  Kudies  using  computer  simulations  which 
verify  die  theoretical  predictions  sad  provide  the  region  of  validity  of  the  analysis.  Further  experi¬ 
ments  study  the  interfering  effect  of  several  dosely  spaced  sinusoids.  The  method  of  overspecified  nor¬ 
mal  equation  is  shown  to  be  muds  more  sensitive  to  this  interference  than  the  ME  method.  Finally, 
same  further  empirical  studies  are  made  of  the  resolution  capabilities  at  the  method.  Using  the  data 
derived,  an  empirical  model  is  derived  which  seems  to  agree  to  some  extent  with  the  data. 


UNCLASSIFIED 


StCUNlTy  CLASSIFICATION  OF  THIS  PAOifWMn  *"»•  f4) 


Overspecified  Normal  Equations 
for  Spectral  Estimation 
by 

David  Izraelevitz 


Submitted  to  the  Department  of  Electrical  Engineering  and  Computer  Science, 
June,  1983,  in  partial  fulfillment  of  the  requirements  for  the  degree  of 

Master  of  Science 


9,,':  5^  ABSTRACT 

There  is  a  one-to-one  relationship  between  a  set  of  F  normalized  positive  definite  correlation  esti¬ 
mates  and  the  P  predictor  coefficients  derived  using  autoregressive  modeling.  Several  researchers  have 
proposed  the  use  of  M  >  P  correlation  estimates  to  provide  a  better  Ptb  order  model.  Specifically,  the 
normal  equations  are  augmented  to  provide  M  linear  equations  between  the  correlation  estimates  and 
die  predictor  coefficients.  Since  die  system  of  equations  is  now  overspecified,  a  least  squares  solution 
is  required. 

In  this  thesis  a  study  is  presented  of  some  of  the  properties  of  the  method  of  overspecified  nor¬ 
mal  equations  as  applied  to  the  problem  of  spectral  estimation.  The  main  contribution  of  this  thesis  is 
the  derivation  of  the  relationships  between  die  number  of  correlations  used,  the  model  order  and  the 
signal  to  noise  ratio  of  the  signal,  to  the  characteristics  of  the  resulting  spectral  estimate.  The  charac¬ 
teristics  studied  are  the  spectral  height,  bandwidth  and  area.  The  method  is  shown  to  be  a  spectral  den¬ 
sity  estimator  hire  the  ME  method,  where  spectral  areas  rather  than  spectral  values  should  be  inter¬ 
preted  as  estimates  of  power.  v: - 

The  relationships  derived  point  to  the  number  of  correlations  used  over  the  minimum,  i.e.  model 
order,  as  an  sgnal-to-noiae  enhancer.  The  resulting  spectrum  is  equivalent  to  the  ME  spectrum  under 
higher  signal-to- noise  conditions.  Another  result  is  the  requirement  of  a  proportionality  constant 
dependent  on  the  number  of  correlations  and  the  model  order  which  is  necessary  for  unbiased  signal- 
to- noise  measurements.  This  constant  is  not  required  however,  for  measurements  of  relative  power 
within  the  same  qpectral  estimate,  as  in  the  power  ratio  of  two  sinusoids  in  noise. 

The  second  part  of  the  them  presents  some  empirical  studies  using  computer  simulations  which 
verify  the  theoretical  predictions  and  provide  the  region  of  validity  of  the  analysis.  Further  experi¬ 
ments  study  the  interfering  effect  of  several  closely  qiaced  sinusoids.  The  method  of  overspecified  nor¬ 
mal  equations  is  shown  to  be  much  more  sensitive  to  this  interference  than  the  ME  method.  Finally, 
same  further  empirical  studies  are  made  of  the  resolution  capabilities  of  the  method.  Using  the  data 
derived,  an  empirical  model  is  derived  which  seems  to  agree  to  some  extent  with  the  data. 
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CHAPTER  1 
INTRODUCTION 


1.1  Motivation 

A  very  important  problem  in  signal  processing  is  that  of  information  "extrapolation."  For  exam¬ 
ple,  what  may  be  available  is  a  small  set  of  measurements  from  which  it  is  desired  to  make  some  rea¬ 
sonable  guess  of  what  mare  measurements  would  have  given.  By  necessity  some  restrictions  must  be 
imposed  on  the  postulated  structure  of  the  data  in  order  to  restrict  its  possible  behavior.  Otherwise,  it 
is  not  possible  to  predict  future  measurements  with  confidence.  The  restrictions  imposed  on  the  data 
under  study  comprise  a  model.  A  model  can  be  thought  of  as  a  structure  for  the  data  with  some  free 
parameters  which  are  then  tuned  to  the  specific  situation.  Although  the  structure  may  seem  too  res¬ 
trictive,  it  is  hoped  that  by  varying  the  free  parameters  a  reasonable  representation  can  be  ascertained. 

A  model  commonly  applied  to  time  series  data  is  the  autoregressive  (AR)  model.  The  basic 
tenet  of  this  model  is  that  the  signal  under  study  is  composed  of  a  linear  combination  of  previous  signal 
values  phis  an  independent  white  noise  sequence  which  allows  for  some  uncertainty  in  the  actual  value 
the  signal  will  take  on  at  any  given  time.  Hie  number  and  values  of  the  weighing  coefficients  of  previ¬ 
ous  samples,  and  the  power  of  the  noise  sequence,  are  the  means  by  which  we  can  tune  and  summarize 
the  signal’s  structure. 

The  literature  is  rich  with  methods  of  extracting  these  model  parameters  from  a  data  segment, 
[10].  Most  of  these  can  be  considered  as  implementations  of  the  maximum  entropy  (ME)  method 
introduced  by  Burg,  [1],  for  the  case  of  one  dimensional  signals  with  given  contiguous  correlations. 
For  this  case,  the  ME  method  exploits  a  matrix  relationship  between  the  autocorrelation  function  of  an 
AR  sequence  and  the  weighing  coefficients  of  the  model.  Specifically,  there  is  a  one-to-one  mapping, 
up  to  a  scale  factor,  between  the  first  P  correlation  coefficients  and  the  P  weighing  coefficients  for  a 
/*th  order  model.  Different  methods,  such  as  the  autocorrelation  and  covariance  methods,  differ  on 
how  the  autocorrelation  sequence  is  estimated  from  the  data. 
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There  has  been  much  attention  given  recently  to  a  new  method  for  AR  modeling  introduced  by 
Cadzow,  [2],  which  can  be  considered  as  an  extension  of  the  classical  AR  or  ME  methods  described 
above.  This  new  method  will  be  referred  to  in  this  thesis  as  the  overspecified  normal  equation  (OSNE) 
method.  The  method  proposed  uses  many  more  correlation  estimates  than  the  order  of  the  model  pos¬ 
tulated.  Although  this  method  was  originally  proposed  for  autoregressive  moving  average  (ARMA) 
modeling,  several  authors  have  recently  noted  the  advantages  of  the  OSNE  method  for  purely  autore¬ 
gressive  modeling,  [7],  [3].  Among  these  advantages  is  the  method's  improved  resolution  of  spectral 
peaks  for  the  same  model  order  as  the  ME  method,  while  being  less  sensitive  to  correlation  estimate 
inaccuracies  than  larger  order  ME  model  estimates. 

The  ME  and  OSNE  methods  have  been  applied  to  the  problem  of  estimating  pure  tones  in  noise. 
The  procedure  followed  is  to  first  extract,  using  the  ME  or  OSNE  methods,  an  AR  model  of  the  sig¬ 
nal.  From  this  model  the  power  spectrum,  or  the  Fourier  transform  of  the  signal  autocorrelation 
sequence,  is  estimated.  Finally,  using  this  power  spectrum  estimate,  the  important  characteristics  of 
the  sinusoids  such  as  frequency  location  and  power  are  extracted. 

The  family  of  signals  of  sinusoids  in  noise  is  of  much  importance  in  the  study  of  monochromatic 
plane  waves  being  measured  by  an  array  of  sensors,  [5].  These  signals  also  provide  a  means  of  charac¬ 
terizing  some  properties  of  spectral  estimation  methods,  such  as  frequency  bias  and  the  ability  to 
resolve  two  sinusoids  very  close  in  frequency. 

As  noted  earlier,  the  link  between  the  data  modeling  techniques  and  the  estimation  of  sinusoids 
in  noise  is  provided  by  the  power  spectrum.  It  has  been  shown  that  sinusoids  in  noise  cannot  be 

i 

represented  by  a  finite  order  AR  model,  [14].  Therefore,  the  power  spectral  estimate  that  is  extracted 
will  never  be  that  of  sinusoids  in  noise  as  required.  In  this  case,  some  sort  of  guidelines  must  be  pro¬ 
vided  to  interpret  the  power  spectrum  that  is  calculated;  how  to  find  the  sinusoidal  power,  for  exam¬ 
ple.  This  problem  was  studied  for  the  ME  case  by  Lacoss,  [8],  where  he  showed  that  the  ME  spectrum 
should  be  regarded  as  a  spectral  density  estimator;  that  is,  areas  underneath  spectral  peaks,  rather  than 
the  peak  values  themselves  are  proportional  to  the  power  in  the  spectral  region. 
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In  this  thesis,  the  spectral  characteristics  of  the  OSNE  method  are  studied.  This  work  provides 
the  necessary  link  by  which  power  spectral  estimates  using  the  OSNE  method  can  be  properly  inter¬ 
preted  to  estimate  sinusoid  power.  In  addition,  the  behavior  of  the  spectral  estimate  is  studied  as  a 
function  of  the  parameters  of  the  algorithm  and  the  characteristics  of  the  signal;  the  order  of  the  model 
used,  the  number  of  correlations  used,  and  the  signal-to-noise  ratio.  Theoretical  results  axe  presented 
far  the  case  of  a  single  complex  exponential  in  noise.  These  results  are  then  generalized  under  some 
conditions  to  several  real  sinusoids  in  noise.  The  validity  of  the  mathematical  approximations  and 
assumptions  in  the  analysis  are  studied  uong  computer  simulations  on  ideal  correlation  estimates. 
Finally,  empirical  studies  are  made  of  the  resolution  properties  of  OSNE,  that  is,  under  what  condi¬ 
tions  of  signal-to-noise  ratio  and  frequency  separation,  the  OSNE  method  will  be  able  to  detect  two 
distinct  sinusoids  instead  of  one. 


ft 


ft 
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1.2  Thesis  outline 

The  classic  results  is  spectral  analysis  using  AR  modeling  are  introduced  and  discussed  in  chapter 
2.  The  normal  equations  for  the  estimation  of  the  model  parameters  are  derived.The  well-known 
results  by  Lacoss  on  the  shape  of  the  AR  spectrum  for  the  estimation  of  sinusoids  in  noise  are 
presented.  Following  a  discussion  of  the  paper  by  Sa  tori  us  and  Ziedler,  the  Lacoss  results  are  general¬ 
ized  to  several  sinusoids  in  noise.  Finally,  the  method  of  overspecified  normal  equations  due  to  Cad- 
20w  is  introduced. 

The  bulk  of  the  contributions  of  this  thesis  are  presented  in  chapter  3.  Here  the  main  theoretical 
results  concerning  the  spectral  shape  of  the  OSNE  spectrum  are  derived  and  discussed. 

Chapter  4  contains  all  the  computer  simulations  which  verify  the  results  presented  in  chapter  3. 
Furthermore,  empirical  studies  are  presented  concerning  the  resolution  capabilities  of  OSNE. 

Chapter  S  summarizes  the  main  results  of  the  thesis  and  provides  the  conclusions  that  can  be 
drawn  about  the  behavior  of  OSNE  as  a  spectral  estimator. 
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CHAPTER  2 
BACKGROUND 


2.1  Introduction 

In  this  chapter  we  review  some  of  the  dasnc  results  in  spectral  analysis  using  AR  modeling.  The 
presentation  begins  with  a  brief  derivation  of  the  «»i  method  of  extracting  model  parameters  from 
a  data  sequence.  This  method  can  be  considered  as  an  implementation  of  the  maximum  entropy  (ME) 
method  introduced  by  Burg  [1],  for  the  case  of  one  dimensional  signals  with  given  contiguous  correla¬ 
tions. 

We  then  restrict  our  study  to  the  spectral  analysis  of  sinusoids  buried  in  white  noise.  This  will 
allow  for  the  description  of  the  behavior  of  different  spectral  estimation  methods  when  directed  toward 
this  widely  encountered  application  of  spectral  analysis.  Here  some  classic  results  by  Lacoss  on  the 
shape  of  the  ME  model  spectrum  far  a  single  complex  exponential  in  noise  are  discussed.  In  the  case  of 
real  data  applications,  isolated  complex  exponentials  are  not  found,  but  rather  appear  as  pairs  at  sym¬ 
metric  locations  about  zero  frequency.  Thus  the  Lacoss  results  need  to  be  generalized  to  several  com¬ 
plex  exponentials.  This  is  done  in  a  discussion  of  the  results  by  Sa  tori  us  and  Ziedler,[13],  where  a 
peak  decoupling  property  at  ME  spectra  is  discussed.  Their  paper  shows  that  for  well  separated 
auousoids,  the  ME  method  estimates  each  peak  separately.  This  will  basically  extend  the  Lacoss  results 
to  the  case  of  well  separated  spectral  peaks. 

After  considering  the  ME  madding  of  spectra,  the  method  of  overspecified  normal  equations, 
first  espoused  by  Cadzow,[2],  far  ARMA  modeling,  is  introduced.  Here  a  large  number  of  correla¬ 
tion  estimates  are  used  to  extract  the  AR  model  parameters. 
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2.2  AR  modeling  and  normal  equations 

Parametric  signal  modeling,  and  more  specifically  autoregressive  (AR)  modeling  of  discrete  time 
sequences,  has  found  many  important  applications  in  such  areas  as  phased  array  radar  [5],  seismic  data 
analysis  [12],  and  speech  processing  [11].  The  basic  philosophy  of  parametric  signal  modeling  is  to 
extract  from  a  finite  data  «egwi«*  the  underlying  characteristics  of  the  signal  as  expressed  by  a  small 
set  of  parameters,  and  therefore  be  able  to  either  compress  the  information  presented  by  this  data,  or 
extrapolate  the  signal,  thus  "enlarging”  the  data  window. 

It  is  in  this  last  ability  that  this  thesis  is  basically  interested.  Since  the  application  at  hand  is  the 
extraction  of  power  and  frequency  characteristics  of  a  sinusoid  corrupted  by  noise,  enlarging  the  data 
window  will  make  the  presence  of  the  sinusoid  more  noticeable,  so  that  we  may  better  estimate  its  fre¬ 
quency  and  power. 

AR  modeling  assumes  that  the  present  signal  sample  is  well  modeled  as  a  tune-invariant  linear 
combination  of  the  last  P  signal  samples  plus  an  independent  white  noise  term.  Uns  structure  may 
seem  restrictive  but  it  has  been  used  successfully  in  the  modeling  of  speech  and  estimation  of  the 
characteristics  of  sinusoids  buried  in  noise.  The  weighing  coefficients  of  previous  samples  and  the 
noise  variance  are  the  characteristic  parameters  which  summarize  the  signal’s  structure. 

Below  is  a  brief  derivation  of  the  relationship  between  the  correlations  of  an  AR  sequence  and 
the  corresponding  model  parameters.  It  is  by  no  means  the  only  derivation  available.  The  reader  is 
referred  to  the  paper  by  Makhoul  [10]  for  an  excellent  treatment  of  the  various  approaches  available 
to  the  AR  mnHiKng  problem.  The  approach  used  here  is  a  statistical  one,  rather  than  a  more  data- 
oriented  approach  of  minimum  prediction  error  for  one  important  reason.  It  is  the  intent  of  the 
derivation  to  decouple  the  relationship  between  exact  correlations  and  model  parameters,  from  the 
important  and  difficult  problem  of  properly  estimating  the  correlation  sequence  from  the  data.  As 
explained  below,  different  correlation  estimates  lead  to  the  autocorrelation  and  covariance  methods. 

According  to  the  AR  model,  the  signal  under  study  *(n]  is  postulated  to  be  of  the  form, 

<•1 


(2.2.1) 


<9 
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where  the  a*  are  the  model  parameters,  also  called  predictor  coefficients,  and  w[n]  is  an  additive, 
zero-mean,  white  noise  sequence. 

Let  us  now  multiply  both  sides  of  (2.2.1)  by  s[#i  -ft]  where  ft>0,  and  then  take  ensemble  expec¬ 
tations. 


*  £{~2oi*[ji— i]*[n— ft]  +  w[n]s[n— ft]}  (2.2.2) 

<•1 

thing  the  linearity  properties  of  the  expectation  operator,  and  defining  the  correlations  as 


=  £{j[ji-/]r[/»-m]}  (2.2.3) 

yields 

£(0,-ft)  =  -2ai*(*-»>”*)  +  £M"]*  ["“*]}  (2-2.4) 

/-l 

We  now  note  that  the  term  £{w[n]j[n-ftJ}  should  be  zero  under  the  model  in  (2.2.1).  The  sig¬ 
nal  sample  j[«-fc]  is  a  function  of  previous  signal  samples,  i.e.,  j[n— ft  — 1],  j[n-fc-2],  etc.  and  the 
noise  term  w[n-ft].  However,  w[n]  is  added  to  the  sequence  after  j[n-ft]  occurs  since  by  assump¬ 
tion  ft>0.  Therefore,  w[#i]  and  j[/i -ft]  are  independent.  Since  w[n]  is  zero-mean,  the  expected 
value  of  the  product  w[n]j[/i—  ft]  should  be  zero  as  well. 

Thus  we  are  left  with  the  following  exact  relationship  between  the  signal  correlations  R  (<  ,ft)  and 
the  predictor  coefficients  ait  the  so  called  normal  equations 

9(0,-*)  -  -2>,J?(-i,-ft)  ft>0  (2.2.5) 

i-i 

There  are  several  important  comments  to  be  made  at  this  point.  The  relationship  expressed  in 
(2.2.5)  is  an  interesting  one,  but  of  little  practical’  value  at  this  stage.  What  it  expresses  is  not  a  rela¬ 
tionship  between  the  data  and  the  model  parameters,  but  rather  between  some  statistical  properties  of 
the  data  sequence  and  the  model  parameters.  Thus  prior  to  applying  this  relationship  the  statistical 
properties  of  r[n]  must  be  awirnnrari  In  other  words  one  of  the  first  steps  in  applying  this  result  will 
be  to  decide  how  R(t,k)  should  be  estimated  from  the  data.  Secondly,  (2.2.5)  is  valid  for  all  ft>0, 
while  there  is  only  a  finite  number  of  unknowns.  Thus  (2.2.5)  is  actually  an  infinite  number  of  simul¬ 
taneous  linear  equations  which  need  to  be  solved  for  the  unknown  at .  Finally,  there  is  a  striking  simi- 


laxity  in  structure  between  (2.2.5)  and  (2.2.1).  By  assuming  that  the  signal  j[n]  is  stationary,  the  rela¬ 
tionship  between  (2.2.1)  and  (2.2.5)  can  be  strengthened.  By  stationarity  we  mean  that  the  statistical 
characteristics  of  the  signal  do  not  change  over  time.  Since  the  statistical  properties  of  the  signal  are 
tune-invariant,  any  statistical  relationship  between  two  signal  samples  must  depend  only  on  the  time 
interval  between  the  two  signals.  In  the  case  of  the  correlation  between  two  signal  samples, 
R(—i,—k),  must  depend  only  cm  (k-i)  at  R(-i  ,-k)  =  R(k-i).  Under  the  additional  assumption 
of  stationarity  then,  (2.2.5)  becomes 

R(k)  =  *>0  (2.2.6) 

i- t 

In  this  case,  the  correlation  sequence  obeys  the  same  recursive  relationship  as  the  signal  sequence 
except  that  the  noise  sequence  is  missing.  Another  difference  is  that  the  recursive  relation  in  (2.2.6)  is 
valid  only  for  k>Q. 

As  noted  previously,  (2.2.5)  and  (2.2.6)  are  actually  overdetermined  sets  of  linear  equations. 
Were  it  not  for  the  assumption  that  the  R(ijt)  used  are  exact,  and  that  j[n]  is  a  Pth  order  AR  pro¬ 
cess,  there  would  probably  be  no  set  of  a,  that  would  satisfy  (2.2.5)  far  all  positive  k.  In  the  classical 
or  ME  approach  to  the  extraction  of  the  predictor  coefficients,  the  above  problem  is  avoided  by  only 
using  (2.2.5)  for  k  in  the  range  of  1  to  P .  Thus  there  are  P  equations  for  P  unknowns  and  a  solution 
far  the  predictor  coefficients  exists  for  any  well-behaved  set  of  *(i  ,k). 

The  ME  method  can  be  expressed  succinctly  in  matrix  notation.  Below,  several  entities  are 
defined  by  specifying  what  each  element  of  the  matrix  or  vector  should  contain 

[*]<,  *  P (-!,-;)  ( P  by  P) 

[r],  -  *(0,-0  (P  by  1) 

(■]<  -  *  (P  by  1) 

Using  the  above  notation,  the  relationship  between  exact  correlations  and  predictor  coefficients 
in  (2.2.5)  can  be  expressed  as 

Ra  «  -r  (2.2.7) 

Different  methods,  such  as  the  autocorrelation  method  and  the  covariance  method,  use  different 

estimates  of  R{i,k)  or  R(k-i),  but  exploit  the  same  relationship  between  the  correlations  and  the 
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predictor  coefficients  as  expressed  in  (2.2.7).  For  example,  the  autocorrelation  method  uses 


N-l-k 

■■  2  *l"M*-*] 

»-0 


as  an  Animate  at  the  stationary  correlation  sequence,  while  the  covariance  method  uses 

as  an  estimate  of  die  correlation  values  where  N  is  the  length  of  the  data  sequence. 


(2.2.8) 


(2.2.9) 


Once  the  predictor  coefficients  have  been  estimated,  the  noise  variance  estimate  is  found  in  turn 
using  (2.2.4)  but  for  the  case  when  *  *  0.  In  this  case  the  value  of  s[/i]  and  w[n]  are  not  uncorre¬ 
lated,  but  have  a  covariance  equal  to  the  variance  of  w[n].  What  results  in  this  case  is  the  following 
relationship  between  R{i  ,k ),  the  a,  set  and  the  noise  variance  a*, 


MO  •  “i«i*(’i.O)  +  al 


(2.2.10) 


After  all  the  model  estimation  calculations  have  been  rrwnplutnH,  the  power  spectral  estimate  for 


j[ii]  can  be  calculated.  If  the  Fourier  transform  of  the  predictor  coefficient  set  is  defined  by 


A(u)  ■  1  +■ 

4*1 


(2.2.11) 


then  the  corresponding  power  spectrum  for  j[«i]  is  given  by 


H(w)|3 


(2.2.12) 


The  basic  idea  in  using  A R  modeling  for  the  estimation  of  sinusoids  in  noise  is  that  we  expect  the 
£(<■>)  given  by  (2.2.12)  to  have  a  large  amount  of  energy  around  the  frequency  of  the  sinusoid  and 
that  the  shape  of  dm  spectrum  around  that  frequency  should  somehow  be  related  to  the  power  in  that 


sinusoid. 
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2.3  Discussion  of  results  by  Lacoss 

Although  the  AR  procedure  is  rather  simple,  requiring  basically  some  evaluations  of 

lagged  products  and  the  solution  of  a  set  of  simultaneous  tiwMr  equations,  what  is  not  obvious  is  how 
to  interpret  the  results.  As  explained  in  chapter  1,  the  power  spectrum  provides  the  link  between  data 
—rtHaimfl  techniques  and  die  estimation  of  properties  of  sinusoids  in  noise.  In  conventional  spectral 
estimation  techniques,  as  Welch's  method,  [15],  in  the  maximum  likelihood  method  (MLM) 
introduced  by  Capon,  [4],  the  value  of  the  spectrum  at  each  frequency  is  interpreted  as  an  estimate  of 
the  power  at  that  frequency. 

hi  a  classic  paper,  Lacoss  has  shown,  [8], that  the  ME  method  behaves  in  a  radically  different 
matitwr  fhan  either  conventional  spectral  estimation  techniques  or  MLM.  The  ME  spectral  estimate 
should  not  be  interpreted  an  a  spectral  value  basis.  The  value  at  a  spectral  peak,  Lacoss  shows,  is  pro¬ 
portional  to  the  square  of  the  sinusoid  signal  power.  However,  the  area  underneath  the  peak,  as 
iMwiiiH  by  the  product  of  the  spectral  peak  value  and  its  half-power  bandwidth,  is  proportional  to  the 
peak  value. 

The  starting  point  of  bis  discussion  is  to  assume  that  we  are  considering  a  complex  sequence  com¬ 
posed  of  a  single  complex  exponential  of  radian  frequency  uq  and  an  additive,  zero-mean,  white  noise 
sequence,  w[n],  i.e. 

i[/»]  =  VbeJm*  +  w[n]  (2.3.1) 

Note  that  j[n]  is  also  zero-mean  as  well  as  stationary.  The  exact  correlation  sequence  for  the  above 

signal  is  given  by 

R(k)  -  8(t)  +  be1*  (2.3.2) 

where  8(i)  is  the  unit  sample  sequence.  If  we  now  insert  the  above  values  for  R(k)  into  equations 

(2.2.7)  and  (2.2.9),  we  can  solve  for  the  at' s  and  w}.  After  inserting  in  (2.2.12),  we  get  the  estimated 

v 

spectrum  to  be 

1  +  b(P  +  1) 

_ 1  +  bP 

U”  j  ^y,//»(‘*>0“«*>)l 


(2.3.3) 
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where 


-  7,4*-'“  -  '^Tsk  (2  i  i> 

The  notation  used  above  is  slightly  different  from  that  used  by  Lacoss  in  his  paper,  however  it  will  sim¬ 
plify  later  comparisons  if  the  results  axe  expressed  in  this  form. 

It  is  readily  seen  that  the  value  of  5(u)  at  <i>0  is 

J(«o)  =  (1  +  bP)(  1  +  b{P  +  1))  a  b2P(P  +  1)  for  bP  »  1  (2.3.5) 

Thus  the  value  of  S  (o>)  at  o>0  is  not  proportional  to  the  power  in  the  sinusoid,  but  actually  proportional 

to  b2.  Also,  we  see  that  the  spectral  maximum  occurs  at  id  =  o>0  so  at  least  for  exact  correlations,  the 

frequency  estimate  is  unbiased.  To  further  characterize  the  shape  of  the  spectral  peak,  La  con  found  an 

approximation  to  the  bandwidth  of  the  spectral  peak  using  a  Taylor  series  approximation  about  the 

peak  value.  This  method  yields  the  following  result  for  the  half-power  bandwidth  of  the  AR  spermr-t 

,  for  large  P, 


(2.3.4) 


“*  =  IFTTjr  « 

Thus  the  bandwidth  of  the  spectrum  decreases  as  the  signal  to  noise  ratio  increases,  i.e.  b ,  as  well  as 
decreasing  as  the  square  of  the  model  order.  Lacoss  proposed  the  product  of  5(oj0)  and  u>#  as  an  esti¬ 
mator  of  the  power  in  the  sinusoid.  This  product  we  see  is  basically  proportional  to  b  independent  of 
model  order. 

From  (2.3.3)  a  bias  in  the  noise  variance  estimate  is  found.  As  b  goes  to  infinity,  while  P  stays 
snail,  the  noise  variance  estimate  does  not  go  to  1,  but  rather  goes  to  1  •+■  y.  This  will  introduce  a 

bias  in  subsequent  calculations,  especially,  as  we  will  find  later,  this  bias  increases  as  the  number  of 
sinusoids  under  study  increases.  This  point  was  not  made  in  the  paper  by  Lacoss,  since  a  bias  in  the 
noise  variance  estimate  will  not  affect  relative  power  measurements,  i.e.,  the  ratio  in  power  between 
two  sinusoids. 

To  avoid  this  bias,  subsequent  discussions  on  the  AR  model  spectrum  will  concentrate  on  the 
"normalized"  spectrum  SN(w)  given  by 


9 
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2.4  Discussion  of  spectral  peak  decoupling  property 

It  is  important  to  keep  in  mind  in  interpreting  the  Lacoss  results,  that  the  signal  under  con- 
hderation  is  a  complex  expoueutial.  Real  sinusoids  can  be  thought  of  as  pairs  of  complex  exponentials 
and  therefore  we  mutt  be  careful  to  examine  where  and  how  the  results  discussed  above  apply  to  real 
signals. 

Although  Larots  studied  the  interfering  effects  of  multiple  complex  exponentials  on  his  results,  a 
more  elegant  method  was  proposed  by  Satorius  and  Ziedler,[13],  for  studying  the  characteristics  of 
the  ME  spectrum  for  several  sinusoids  again  assuming  that  exact  correlations  are  available.  Using  the 
method  of  undetermined  coefficients,  which  will  be  discussed  in  detail  in  chapter  3,  they  showed  that 
A(w)  in  (2.2.12)  for  a  signal  consisting  of  L  complex  exponentials  located  at  wj,  wL 

with  power  bit  ....  bL  is  given  by 

A(«)  =  (2-4-1) 

/-I 

where  )  is  the  same  defined  in  (2.3.4)  and  the  dt  are  unknown  constants.  An  important  property 

an  JC 

to  take  note  here  is  that  />( X)  is  a  (sine)  type  function  which  goes  to  zero  as  X  increases.  Thus 

A  (u>)  is  a  weighed  sum  of  sine  functions  centered  at  the  frequency  of  each  sinusoid.  The  constants  di 
provide  the  weighing  of  each  sine  function.  Therefore,  for  sufficiently  well  separated  w’s,  (2.4.1)  will 
look  like 

A(u>)  *  l-/,(w-«()4  (2.4.2) 

far  to  =  to,  which  is  independent  of  both  the  number  of  sinusoids  L  and  their  frequencies.  However, 

if  the  value  of  d(  depended  on  the  characteristics  of  the  other  exponentials,  the  decoupling  would  not 

occur.  Fortunately,  the  paper  by  Satorius  and  Ziedler  shows  that  for  well  separated  sinusoids,  the 

value  of  dt  doesn’t  depend  on  the  location  or  power  of  the  other  sinusoids.  In  this  case,  a  simple 

interpretation  of  the  dt  is  available.  Consider  (2.4.2)  for  the  ea«g  when  id=<d(.  Then,  A  (u>)  will  equal 

1— dt.  Thus,  as  d(  gets  closer  to  1,  the  spectrum  will  display  a  peak  at  w, . 

It  is  the  variations  of  A  (u)  which  provide  the  fluctuations  and  detail  as  a  function  of  to  *01  the 
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AR  spectrum.  The  estimate  of  the  noise  variance  is  only  a  proportionality  constant.  Thus  for  well 
separated  frequencies,  equation  (2.4.1)  extends  the  Lacoss  results  to  the  shape  of  each  individual  spec¬ 
tral  peak. 

As  discussed  previously,  the  noise  variance  estimate  is  biased  for  the  single  complex  exponential 
in  noise  case.  For  several  sinusoids  in  noise,  the  pioblem  is  even  worse.  For  well  separated  sinusoids, 
the  noise  variance  is  given  by 


m  1  +  2' 


(2.4.3) 


ifxbiP  +  1 

As  P  goes  to  infinity,  the  value  of  e*  goes  to  the  right  value  1,  but  as  bt  goes  to  infinity,  a*  goes  to 


=  l  +  J  (2.4.4) 

which  deviates  from  1  more  and  more  as  the  number  of  sinusoids  being  estimated  increases.  What  we 
can  expect  from  (2.4.4)  is  that  the  measure  proposed  by  Lacoss  for  the  signal  power  will  be  off  by  a 
constant  amount  as  the  SNR  goes  to  infinity.  For  this  reason,  the  estimation  of  the  noise  power  will 
not  be  dealt  with  further  in  this  thesis,  since  computer  experiments  have  shown  that  this  bias  exists  in 
the  OSNE  spectrum  as  well.  Without  an  absolute  power  measurement,  we  are  forced  to  restrict  our 
measurement  to  that  of  relative  power,  that  is  power  ratio  between  two  sinusoids  in  the  same  spec¬ 
trum,  or  the  calculation  of  signal  to  noise  ratio. 
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2.5  Overspecified  normal  equations  (OSNE) 

In  this  section  we  will  consider  a  different  use  of  the  equations  derived  in  section  2.2  for  the  pur¬ 
pose  of  spectral  estimation.  The  key  observation  to  make  is  that  the  relationships  between  the  true 
correlations  and  the  predictor  coefficients  expressed  in  (2.2.5)  and  (2.2.6)  hold  not  only  for  k  between 
1  and  P ,  but  really  hold  far  all  positive  k .  As  described  previously,  the  ME  method  uses  just  enough 
simultaneous  linear  equations  to  make  the  solution  unique. 

If  the  model  used  is  the  right  one,  that  is  if  j[r]  is  a  Pth  order  AR  process,  and  if  the  correla¬ 
tions  are  estimated  exactly,  then  the  P  equations  used  in  the  ME  method  are  all  that  is  needed.  Subse¬ 
quent  equations,  those  in  (2.2.5)  for  k>P ,  will  not  add  any  more  information. 

Of  course  in  real  applications,  neither  assumption  is  satisfied.  In  fact,  even  if  we  are  quite  confi¬ 
dent  in  using  a  Pth  order  AR  model  for  the  signal  under  study,  the  correlations  will  never  be  estimated 
exactly.  Thus,  we  can  be  sure  that  the  higher  index  equations  will  certainly  provide  different,  although 
possibly  not  better,  information  regarding  the  predictor  coefficients. 

The  method  of  overspecified  normal  equations  can  therefore  be  stated  as  finding  a  set  of  predic¬ 
tor  coefficients  which  will  minimize  in  some  way  the  difference  between  the  two  sides  of  (2.2.5)  or 

(2.2.6)  for  the  ca«e  of  inexact  correlation  estimates  or  inaccurate  model.  If  the  maximum  available 
index  difference  in  the  correlation  estimate  is  M ,  then  the  number  of  available  linear  equations  in 
(2.2.5)  or  (2.2.6)  is  M.  To  provide  some  mathematical  tractability,  a  quadratic  norm  of  the  differ¬ 
ence  between  the  left  and  right  sides  of  (2.2.5)  or  (2.2.6)  is  used  as  the  function  to  be  minimized. 
Additionally,  Cadzow  originally  proposed  the  use  of  a  weighing  function,  arguing  that  it  would  be 
expected  that  higher  correlation  estimates  would  be  less  accurate  than  lower  correlations,  hence  imply¬ 
ing  that  the  differences  in  (2.2.5)  for  equations,  using  higher  correlations  should  be  weighed  less  in  the 
objective  function. 

We  can  again  express  this  problem  succinctly  in  matrix  notation.  The  same  notation  as  in 

(2.2.7)  will  be  used  to  define  R,  r,  and  a.  Let  the  P  by  M  matrix  IM/>  be  defined  as  the  concatenation 
of  a  P  by  P  identity  matrix  and  a  P  by  ( M—P )  zero  matrix.  That  is 
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(3/  by  P)  (2.5.1) 

Hie  basic  property  of  this  matrix  is  that  it  will  add  M-P  zeroes  to  a  vector  of  dimension  P .  Follow* 
ing  tins  notation,  I/p  is  a  P  by  P  identity  matrix. 

The  OSNE  problem  can  be  phrased  as  finding  the  least  squares  solution  to 

RIj*a  a  -r  (2.5.2) 

In  this  case  the  dimension  of  R  in  (2.5.2)  is  M  by  Af  while  in  (2.2.7)  it  was  P  by  P .  Similarly,  the  r 

vector  is  now  M  by  1.  The  effect  of  the  matrix  can  be  interpreted  as  either  attaching  M-P 

zeroes  to  a  or  'chopping  off”  the  last  M-P  columns  of  R.  According  to  the  Orthogonality  Principle, 

the  least  squares  solution  to  (2.5.2)  must  satisfy 

I^RR^Wa  =  -I&R"r  (2.5.3) 

In  the  above  equation  the  superscript  H  denotes  the  Hennitian  operation  on  the  corresponding  matrix. 

The  use  of  a  different  number  of  predictor  coefficients  and  equations  permits  a  trade-off  between 
resolution  and  statistical  stability  [7].  It  has  been  observed  empirically  that  using  M  correlations  and  a 
Pth  order  model  yields  resolution  capabilities  which  lie  somewhere  between  using  a  P  th  order  model 
and  an  M th  order  model  by  the  ME  method. 

As  is  usually  the  case  in  spectral  estimation  methods,  there  is  an  inherent  tradeoff  in  statistical 
stability  and  spectral  resolution.  That  is,  in  general  the  higher  resolution  an  algorithm  has,  the  more 
sensitive  it  is  to  the  accuracy  of  the  correlation  measurements.  Thus  any  variation  in  the  correlation 
estimates  is  magnified  in  the  resulting  spectrum  for  higher  resolution  algorithms.  In  adherence  to  this 
general  principle,  the  OSNE  spectrum  using  a  P  th  order  model  and  M  equations  is  found  to  be  more 
statistically  stable  than  the  ME  spectrum  using  an  Af  th  order  model,  but  is  less  statistically  stable  than 
the  ME  spectrum  using  an  P  th  order  model,  in  opposite  relation  to  the  resolution  performance. 

The  above  statements  on  statistical  stability  have  been  purposely  vague;  their  intent  is  to  give  a 
qualitative  assessment  of  the  stochastic  behavior  of  the  algorithms.  Of  course  if  the  stochastic  behavior 
of  the  OSNE  method  were  to  be  considered  in  depth,  a  definition  would  be  needed  for  the  term  statist¬ 
ical  stability,  whether  this  refers  to  variability  in  spectral  peak  location,  or  model  parameter  value,  or 
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CHAPTER  3 

BEHAVIOR  OF  OSNE  AS  A  SPECTRAL  ESTIMATOR 

3.1  Introduction 

As  explained  in  previous  sections,  the  power  spectral  estimate  provides  the  link  between  data 
mnrirfing  techniques  and  the  estimation  of  sinusoids  in  noise.  The  purpose  of  this  thesis  is  to  develop 
the  link  connecting  the  OSNE  method  and  the  sinusoid  in  noise  problem  in  analogy  to  the  work  by 
Lacoss,  Satorius  and  Ziedler  for  the  ME  method. 

To  provide  the  proper  background  to  interpret  the  OSNE  spectrum  the  following  questions  need 
to  be  answered, 

1.  Does  OSNE  behave  like  conventional  spectral  estimates  where  spectral  values  correspond  to  spectral 
power,  or  does  it  behave  like  the  ME  method  where  spectral  areas  correspond  to  power? 

2.  How  do  spectral  characteristics  such  as  peak  value  and  bandwidth  vary  as  a  function  of  algorithm 
and  signal  parameters;  that  is,  the  model  order,  the  number  of  correlations  used,  and  the  sinusoid 
powers? 

3.  Under  what  conditions  can  separate  peaks  be  treated  individually;  that  is,  does  OSNE  satisfy  the 
property  discussed  by  Satorius  and  Ziedler  for  the  ME  method? 

In  this  chapter  the  theoretical  results  are  developed  to  help  answer  these  questions. 
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3.2  Analysis  of  the  OSNE  spectral  estimate 

The  problem  to  be  discussed  in  this  section  is  the  description  of  the  OSNE  spectrum  for  the  case 
of  several  sinusoids  in  noise.  The  problem  will  be  considered  both  fox  the  case  of  a  single  complex 
exponential  in  noise  and  the  case  of  several  sinusoids.  The  discussion  begins  with  the  consideration  of 
the  OSNE  spectrum  for  several  sinusoids  and  the  decoupling  property  of  the  OSNE  spectral  analysis  is 
derived.  This  will  motivate  the  subsequent  discussion  of  the  OSNE  spectrum  of  a  single  complex 
exponential  in  noise.  In  considering  the  single  complex  exponential  case,  a  fundamental  result  will  be 
found  linking  the  characteristics  of  the  ME  spectral  peak  and  the  OSNE  spectral  peak. 

Consider  the  case  where  the  exact  correlations  for  L/2  sinusoids  in  noise  are  available.  This  is 
the  same  case  considered  in  the  Sa  tori  us  and  Ziedler  paper.  The  radian  frequency  of  each  sinusoid  will 
be  denoted  by  <1^ ,  and  the  corresponding  amplitude  by  Since  sealing  both  sinusoid  powers  and 

noise  power  by  the  same  amount  does  not  affect  the  calculation  of  the  a,  set,  the  noise  variance  will  be 
anuned  to  equal  1. 

hi  the  above  situation,  the  correlation  between  a  data  sample  at  n  and  a  data  sample  at  n  +  it  is 
independent  of  n  and  is  given  by 

in 

*(*)  =  5(4)  +  22*<cos  (3.2.1) 

(«i 

Since  a  cosine  function  consists  of  two  complex  exponentials,  (3.2.1)  can  be  rewritten  as  a  sum  of 

weighed  complex  exponentials.  Letting  u  L  =  -u>,,  and  b  L  =  ,  we  get  an  equivalent  expression 

'*2  ,+J 

far  the  correlation  sequence  of  L/2  sinusoids  in  noise, 

*(4)  =  8(4)  +  £*,•*'“'*  (3.2.2) 

i-i 

We  see  in  this  case  that  the  agnal  under  study  is  stationary  and  therefore  the  formulation  in  (2.2.6) 
will  be  required  to  find  the  OSNE  AR  model. 

The  next  step  is  to  find  the  matrix  R  and  the  vector  r  as  defined  in  (2.2.7)  corresponding  to  the 
autocorrelation  sequence  in  (3.2.2).  To  exprem  the  above  relationships  in  matrix  notation  some  auxili¬ 
ary  matrices  will  have  to  be  Let  the  ht  by  L  matrix  be  defined  as  follows, 


We  see  that  E*  is  composed  of  powers  of  the  complex  exponentials  corresponding  to  each  frequency 
uj.  The  L  by  L  diagonal  matrix  B  contains  the  power  in  each  complex  exponential,  i.e., 

B  =  diag(bltb2,.  ■  ■  ,bL)  (3.2.4) 

Finally,  let  the  L  by  1  vector  j  consist  of  all  ones.  With  all  the  notational  machinery  defined  above, 

we  can  now  expres  succinctly  the  matrix  R  defined  in  section  2.2  corresponding  to  (3.2.2), 

R  -  I**  +  E*BEH*  (3.2.5) 

Similarly,  the  vector  r  can  be  expressed  as 

r  =  E*BJ  (3.2.6) 

After  inserting  (3.2.5)  and  (3.2.6)  into  (2.5.2)  we  can  state  the  OSNE  problem  for  sinusoids  in  noise 

given  the  sinusoid  locations  and  amplitudes, 

(I**  +  E*BE*m)W.  *  -E*Bj  (3.2.7) 

The  obvious  next  step  would  be  to  solve  directly  for  the  a,  set  using  (2.5.3).  However,  inverting  the  R 

matrix  is  a  difficult  task  except  for  the  simple  case  of  L  =  1  where  the  Woodbury  identity  could  be 

used.  A  more  insightful  approach  was  introduced  by  Satorius  and  Ziedler  and  latex  used  by  Lang  [9] 

which  is  similar  in  philosophy  to  the  method  of  undetermined  coefficients  used  in  solving  ordinary 

linear  differential  equations,  hi  this  situation  we  make  the  assumption  that  the  solution  to  (2.5.2)  is 

given  by  a  linear  combination  of  the  ryihirrm«  of  E^ .  Specifically, 

■  =-7E,d  (3.2.8) 

where  d  is  an  L  by  1  vector  and  E^  has  the  same  form  as  E*  but  truncated  at  the  Fth  row.  Substitut¬ 
ing  (3.2.5),  (3.2.6)  and  (3.2.8)  into  the  least  squares  solution  condition  expressed  in  (2.5.3)  yields 
(after  some  tedious  manipulation)  an  optimality  condition  for  the  unknown  vector  d, 

Ep[jlu  +  2BF,  +  A#BF*BF,]d  =  E,[IU  +  A/BF*]Bj  (3.2.9) 

where  the  newly  introduced  L  by  L  matrices  Fp  and  F*  are  the  matrix  equivalent  of  the  function  fp{) 
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defined  in  (2.3.4),  and  are  given  by 


Eq«Eq 


Considering  each  element  of  Ffi ,  we  find  that 


(3.2.10) 


IF0]„  =  Zq  (to/ -toy) 

The  above  equation  can  be  amplified  by  noting  that  this  expression  is  equivalent  to  requiring  that  the 
expression  below, 

(“1^  +  2BF,  +  A/BF*BF,]d  -  [Itt  +  A#BF„]Bj  -  n  (3.2.11) 

be  in  the  null  space  of  Ef,  i.e., 


E,n  =  0  (3.2.12)  ; 

Since  E/>  is  a  Vandermonde  matrix,  the  null  space  of  E/>  consists  of  the  zero  vector  only  as  shown  in 

[6].  Therefore,  (3.2.9)  is  simplified  (somewhat)  to  the  following  relation  for  the  unknown  d  vector, 

[yl u  +  2BF/i  +  AfBFtfBF/.Jd  =  [1^  +  AfBF^JBj  (3.2.13) 

It  is  from  studying  (3.2.13)  that  the  properties  of  the  OSNE  spectrum  will  be  derived. 

Recalling  that  the  matrix  B  is  diagonal  and  that  M  and  P  are  scalars,  we  see  that  the  interaction 
between  the  different  elements  of  the  vector  d  occurs  via  the  two  matrices  with  non-zero  elements  in 
the  off-diagonals,  F,  and  F*.  As  both  hi  and  P  are  increased,  these  two  matrices  approach  the  iden¬ 
tity  matrix  I u  (consider  the  relationship  in  (3.2.10)).  Thus  for  large  enough  P ,  each  value  of  dt  will 
be  calculated  from  an  algebraic  relation  which  will  depend  only  on  M ,  P,  and  bt.  The  location  and 
amplitude  of  any  other  sinusoid  will  not  affect  the  value  of  this  dt.  This  is  similar  to  the  result  for  ME 
diactmed  in  section  2.4.  However,  this  does  not  mean  that  the  vector  a  win  in  any  way  reflect  this 
decoupling.  Each  a,  is  a  linear  combination  of  all  the  elements  of  d,  as  well  as  depending  on  the  loca¬ 
tions  of  each  sinusoid  via  the  matrix  Ef.  As  we  wifi  find  later,  considering  the  d  vector  rather  than 
the  set  of  predictor  coefficients  win  be  a  more  natural  choice  for  examining  the  properties  of  the 
OSNE  spectrum. 

This  decoupling  property  of  the  elements  of  d  leads  directly  to  the  decoupling  of  the  spectral 
peaks.  Recan  that  the  predictor  Fourier  transform  is  given  by 
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<* 


Cl 


M 


A(<u)  =  1  —  2*  -'ulak  (3.2.14) 

*»i 

Substituting  the  dements  of  d  for  the  ak  (3.2.14)  becomes 

A(U)  =  l-i/,(o»-WlK  (3.2.15) 

i-i 

where  the  function  /,( )  is  the  now  familiar  friend  previously  defined.  Equation  (3.2.15)  is  the  same 
as  (2.4.1)  where  the  property  of  the  ME  spectrum  was  discussed.  Following  the  same  rea¬ 

soning  as  proposed  in  section  2.4,  for  weD  separated  sinusoids, 

A(«)  =  1  -/Xw-WiM  (3.2.16) 

for  <a  ~  <■>,-.  Again,  it  must  be  emphasized  that  increasing  the  number  of  correlations  used,  i.e.  M , 

alone  is  not  enough  to  decouple  the  spectral  peaks.  The  value  of  P  must  be  large  enough  for  two  rea¬ 
sons;  first,  to  make  sure  that  <a<)  in  (3.2.16)  decreases  fast  enough,  and  second,  to  decouple  the 
calculation  of  each  dt  in  (3.2.13).  Obviously,  increasing  P  will  force  an  increase  in  M . 

Considering  (3.2.16)  further,  we  see  that  for  well  separated  sinusoids,  d,  defines  the  shape  of  the 
peak  at  This  value  of  d,  is  calculated  as  if  the  data  consisted  of  only  one  complex  exponential  at 
u ,  of  power  b, .  This  leads  us  to  study  in  detail  the  case  of  a  single  complex  exponential  in  noise, 
rince  any  results  which  are  derived  in  this  case  are  applicable  under  some  conditions  to  multiple 
■miioids. 


hi  the  case  of  a  single  complex  exponential  in  noise,  L  equals  1,  the  d  vector  is  a  scalar  dx,  the 
matrix  B  is  a  scalar  bx  and  the  and  F*  matrices  become  the  scalar  1.  The  value  of  dx  is  now  easily 
found, 


ix  = 


(1  +  Mbx)bx 


1  +  Mbx 


Mb , 


j  +  2h,  +  Mb}  ~  +  2  + 

As  a  check  far  the  above  relation,  setting  M  =  P  yields 

d  b'P~ 

*'  ~  l  +  bxP 

Comparing  this  result  with  the  ME  spectrum  found  in  (2.3.3),  we  see  that  they  agree. 


(3.2.17) 


(3.2.18) 


For  large  model  order  P  or  large  si gnal-to- noise  ratio  (SNR)  bx,  a  rather  interesting  property 
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appears;  the  value  of  dx  becomes  dependent  only  on  the  SNR  and  the  number  of  correlations  used. 
For  large  model  order  F  or  for  large  signal  power  b j,  we  can  make  the  following  approximations, 

1 +Mbx  a  Mbx  (3.2.19) 

Thus 

dl  ~  for  *i »1  or  P»  1  (3.2.20) 

1  t  Mo  i 

What  (3.2.20)  implies  is  that  for  large  SNR  or  P,  the  value  of  dx  is  the  same  that  would  have  been 
ralnilafftri  if  instead  of  OSNE,  the  ME  method  had  been  used  with  an  A/th  order  model.  The  reader  is 
warned  not  to  infer  from  this  that  the  spectral  peak  will  have  the  same  shape  for  OSNE  and  ME. 
Recalling  (3.2.16),  the  characteristics  of  A(u»)  near  depend  not  only  on  the  value  of  dx,  but  also  on 
the  behavior  of  fP(  ).  For  an  A/th  order  ME  spectrum,  the  function  would  be  fu{  ),  not  fP(  ).  It  is 
an  interesting  behavior  of  the  OSNE  spectrum  that  the  effect  of  the  model  order  used  is  reflected  in 
the  function  fP{  ),  while  the  effect  of  the  number  of  correlations  used  and  SNR  is  reflected  in  the 
value  of  dx  solely.  Also  we  can  deduce  some  trade-offs  that  OSNE  provides  between  the  number  of 
correlations  and  the  SNR.  Suppose  that  we  consider  two  situations  where  the  model  order  used  is  the 
same;  first,  the  SNR  of  the  signal  is  bx  and  the  number  of  correlations  used  M\  second,  the  SNR  of 

the  signal  is  y  while  the  number  of  correlations  used  is  Mk .  Considering  (3.2.20)  and  (3.2.16),  we 

see  that  the  value  of  dx  remains  the  same,  while  the  function  fP{  )  is  unaffected.  Therefore,  the  shape 
of  the  spectral  peak  will  be  virtually  the  same  both  in  spectral  value  and  bandwidth.  Thus  the  effect  of 
increasing  the  number  of  correlations  is  to  make  the  effective  SNR  larger. 

The  above  discussion  presented  an  interpretation  of  what  increasing  the  number  of  correlations 
did  in  terms  of  artificially  boosting  the  SNR.  Another  effect  which  is  of  interest  is  to  find  what  trade¬ 
offs  occur  in  varying  the  model  order  as  well  as  the  number  of  correlations  used.  The  discussion  below 
provides  an  interesting  result  which  also  provides  another  link  between  the  ME  and  OSNE  spectra. 

Consider  now  the  A  (w)  corresponding  to  the  Af th  order  ME  spectrum;  that  is,  if  the  M  correla¬ 
tion  values  had  been  used  to  extract  an  Afth  order  model. 
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In  this  case,  from  (2.3.3) 


A(tt)  =  1 "  TT^i7/-(wo_w) 


(3.2.21) 


where  fm{  )  was  previously  as 

.  v  1  x  anM  X  _ , 

/*'(k)  =  '  vS*  <3-2  22) 

The  task  at  hand  is  to  find  a  reasonable  approximation  to  fy(  )  for  small  X.  For  small  X,  sin  X  ~  x, 
so  /w(  )  could  be  considered  to  depend  only  on  the  product  MX  or  (A/+1)X.  Although  both  approxi¬ 
mations  are  valid  for  small  enough  X,  the  second  choice  has  a  wider  region  of  validity  as  shown  in 
Table  3.2.1.  Thus  the  approximation  we  will  use  is  that 

fM  ~  =  /I(M+1)X]  (Af-l)X«l  (3.2.23) 

With  the  approximations  and  assumptions  discussed  above  in  mind,  A  (w)  for  an  A/th  order  ME  spec¬ 
trum  is  approximated  by, 

,  ,  Mh\ 

>W<*0  ~  1  -  i  +  ub/M+W”  "  “.)!  (A/+l)(w— *«),)«  1  (3.2.24) 

The  above  approximations  are  now  applied  to  the  OSNE  predictor  Fourier  transform.  Recall  (3.2. 15) 
for  L  =  1,  which  provides  the  predictor  Fourier  transform  for  the  use  of  OSNE  on  a  single  complex 
exponential  in  noise, 


^gsws(u)  ~ 


1+Mbt 


—  +  2  +  A/b, 


-//,(«-<*>/) 


(3.2.25) 


If  we  now  apply  the  approximation  in  (3.2.20)  for  the  value  of  for  large  P  or  large  SNR,  we  find, 

A/^ 

^osme(“)  =*  1-—  ^^/^((ii-w,)  for  b i  »  1  or  P  »  1  (3.2.26) 

Using  the  approximation  in  (3.2.23)  for  the  behavior  of  />(  ),  a  final  form  is  found  for  the  OSNE 
predictor  Fourier  transform, 


Mb\ 


(3.2.27) 


AoaeOu)  «  1-  j  +‘]j^i71(^+1)(w-(d/)] 

for  b\  »  1  or  P  »  1  and  (F  +  l)(w— u>,)  «  1 

The  above  equation  (3.2.27)  is  one  of  the  fundamental  results  of  this  thesis.  This  equation  shows 


Table  3.2.1 

real  part 

X 

M  approx, 
error 

M+l  approx, 
error 

M  approx, 
error 

M+l  approx, 
error 

ETvmTM 

1.030e-03 

-3.640e-04 

-9.930e-03 

-4.021e-05 

2.000e-02 

4.077e-03 

-1.425e-03 

-1.944e-02 

-3.186e-04 

3.000e-02 

9.016e-03 

-3.090e-03 

-2.814e-02 

-1.059e-03 

4.000e-02 

1.564e-02 

-5.213e-03 

-3.S67e-02 

-2.454e-03 

5.000e-02 

2. 369e-02 

-7.596e-03 

-4.169e-02 

-4.657e-03 

6.000e-02 

3.281e-02 

-l.OOle-02 

-4.596e-02 

-7.769e-03 

7.000e-02 

4.265e-02 

-1.220e-02 

-4.830e-02 

-1.183e-02 

8.000e-02 

5.279e-02 

-1.390e-02 

-4.861e-02 

-1.681e-02 

9.000e-02 

6.282e-02 

-1.485e-02 

-4.685e-02 

-2.263e-02 

Comparison  of  M  and  Ml  *  1  approximations  to  the 
/m(A)  function  in  equation  (3.2.23). 

Error  is  defined  as  approximation  minus  actual  value. 
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dear  ty  bow  each  of  the  parameters  of  tbe  problem,  i.e.,  the  model  order  used,  the  number  of  correla¬ 
tions,  and  signal  power  affect  the  structure  of  the  resulting  power  spectral  estimate. 

Comparing  (3.2.27)  and  (3.2.24)  provides  a  very  interesting  and  useful  correspondence  between 
the  ME  and  OSNE  spectra  near  where  the  peak  is  located.  Note  that  the  only  difference  between  the 
two  equations  is  that  in  (3.2.24)  the  factor  (u-Uj)  is  multiplied  by  (M +1),  while  in  (3.2.27),  (w-w,) 
is  multiplied  by  (P-t-1).  Thus  one  is  just  a  frequency  stretched  version  of  the  other.  Specifically, 

Aosm(PM)(ui  +  A)  =  Ata(tt)  (<*»i  +  (P  +  l)A/(A/+l)j  (3.2.28) 

for  (P+1)A  «  1  and bx  »  1  or  P  »  1 

An  interpretation  of  (3.2.28)  is  that  the  OSNE  spectrum  for  M  correlations  and  Pth  order  model 
is  equivalent  around  the  spectral  peak ,  to  the  spectrum  resulting  from  ME  analysis  using  an  Afth  order 
model,  but  with  a  frequency  stretching  by  a  factor  of  (M  ■+■  1)/(P  1). 

As  discussed  earlier,  the  number  of  correlations  used  and  the  signal  power  affect  the  spectral 
shape  only  as  a  function  of  their  product.  The  model  order,  however,  controls  the  "frequency  scaling" 
of  the  spectrum.  For  example,  suppose  the  signal  power  and  number  of  correlations  are  kept  constant, 
while  the  mnHel  order  used  is  2P +1,  then  the  resulting  spectrum  will  have  tbe  same  maximum  value 
while  the  spectral  peak  will  be  shrunk  in  the  frequency  axis  by  a  factor  of  two.  Therefore,  the  half- 
power  bandwidth  will  be  halved.  The  effect  of  varying  the  model  order  for  constant  M  and  signal 
power  is  shown  in  figure  3.2.1. 

Thus  using  (3.2.28)  and  the  discussion  above,  we  can  generalize  the  Lacoss  results  to  the  case  of 
the  OSNE  spectrum.  Recall  however,  that  unlike  Lacoss,  we  do  not  consider  the  noise  power  estima¬ 
tion,  but  rather  use  just  5* (<■>).  The  spectral  height  then  for  the  OSNE  spectrum  using  M  correlations 
and  P  predictor  coefficients  is  the  unit  as  the  spectral  height  for  ME  using  M  predictor  coefficients,  or 
from  (2.3.7), 

~  5jKe(w0)  ~  b2M2  6,  »  1  or  P  »  1  (3.2.29) 

The  Lacoss  result  on  the  bandwidth  of  the  spectral  peak  can  also  be  generalized  yielding  from  (2.3.6), 


~  4 

~  b(P  +  1)(A/  -  1) 


6,  »  1  or  P  »  1 


(3.2.30) 
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Considering  (3.2.29)  and  (3.2.30),  we  see  that  OSNE  is  a  spectral  density  estimator  since  the  product 
of  spectral  height  and  bandwidth  is  proportional  to  signal  to  noise  ratio.  However,  the  spectral  area  is 
no  longer  independent  of  model  order  or  the  number  of  correlations.  There  is  basically,  for  large  P,  a 
factor  of  M/P  which  doea  not  cancel  out.  Therefore  to  use  OSNE  as  a  power  estimator  the  spectral 
area  must  be  calculated  and  a  proportionality  constant  which  depends  on  both  the  model  order  and  the 
number  of  correlations  must  be  introduced.  Specifically, 

*'<-•>  ■  “» -  <3-2-3» 
This  proportionality  constant  will  have  no  effect  if  the  power  ratio  between  two  sinusoids  in  the  same 

spectrum  is  required  since  the  constant  is  independent  of  the  snusoid  characteristics.  However,  if  an 
SNR  value  is  required,  this  constant  must  be  accounted  for. 

One  final  comment  is  to  note  that  b  is  not  the  power  in  the  sinusoid,  but  rather  the  power  in  the 
complex  exponential.  Thus  for  real  sinusoids,  (3.2.29),  (3.2.30),  and  (3.2.31)  must  be  altered  slightly 

replacing  b  by  y  where  a  is  the  power  in  the  real  sinusoid.  The  above  results  are  summarized  in 

Table  3.2.2,  where  the  Lacoss  results  for  the  spectral  bandwidth  have  been  converted  to  normalized 
radian  frequency. 


9 


Table  3.2.2 


ME 

OSNE 

c?P2 

crM2 

Spectral  peak  of  SN  (ca) 

A 

4 

8 

8 

Spectral  bandwidth  in 
radian  freq. 

p 

S 

4 

>-* 

w 

M 

Cl(P  r  1)(M  T  1) 

Spectral  area 

2 P2 

■■I 

WkBSBSEEm 

Spectral  characteristics  as  a  function  of  the  parameters,  P, 
"  the  model  order,  M,  the  number  of  correlations,  and  a, 
the  signal-to-noise  ratio  for  a  real  sinusoid  in  noise. 

It  is  assumed  that  o  »  1,  or  P  »  1. 
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CHAPTER  4 

EMPIRICAL  STUDIES  OF  OSNE  AS  A  SPECTRAL  ESTIMATOR 

4.1  Introduction 

Hie  previous  chapter  provided  some  theoretical  results  relating  the  values  of  some  algorithm 
parameters  and  the  resulting  behavior  of  the  OSNE  AR  modeling  technique,  when  applied  to  spectral 
estimation.  However,  these  results  only  apply  under  some  conditions,  namely  large  model  orders 
and/or  large  SNR  and  well  separated  spectral  peaks.  The  purpose  of  this  chapter  is  to  study  what  the 
regions  of  validity  are  in  the  above  theoretical  results. 

The  empirical  studies  will  be  performed  on  exact  correlations.  This  requires  some  explanation. 
First,  for  enough  data,  the  correlation  estimates  do  approach  the  exact  correlations,  so  a  study  of  the 
behavior  of  OSNE  under  exact  correlations  becomes  valid  for  the  large  data  case.  Second,  the  use  of 
real  data  introduces  some  stochastic  properties  to  the  problem,  i.e.,  the  results  will  begin  to  vary  sta¬ 
tistically.  This  variation  will  cloud  the  underlying  relationships  among  the  algorithm  parameters. 
Third,  the  use  of  exact  correlations  has  provided  some  rather  elegant  relationships  among  the  algo¬ 
rithm  parameters,  which  provide  at  least  a  qualitative  feel  for  their  interaction.  It  is  important  to  find 
where  these  elegant  relationships  are  valid. 

In  the  first  set  of  experiments,  we  will  consider  the  region  of  validity  of  the  results  of  chapter  3 
as  summarized  in  Table  3.2.1.  Actual  values  for  peak,  bandwidth  and  spectral  area  are  plotted  versus 

predicted  values  for  a  single  complex  exponential  located  at  =  However,  the  choice  of  the  loca¬ 
tion  of  the  complex  exponential  is  completely  arbitrary  since  there  are  no  other  interfering  complex 
exponentials.  These  experiments  study  the  approximations  used  in  deriving  the  basic  results  in  (3.2.27) 
and  (3.2.28). 

The  second  set  of  experiments  considers  the  interaction  of  sinusoids  and  therefore  provides  some 
feel  for  the  peak  coupling  that  occurs  when  the  peak  separation  is  too  small.  One  experiment  studies 
the  interference  on  the  estimation  of  the  characteristics  of  a  sinusoid  by  the  power  at  negative  frequen¬ 
cies.  This  interference  becomes  more  pronounced  as  the  location  of  the  sinusoid  approaches  zero 


frequency.  This  will  show  under  what  situations  the  single  complex  exponential  results  become  valid. 
A  second  experiment  studies  the  resolution  capabilities  of  OSNE.  For  a  given  P ,  M  and  SNR  the 
minimum  spectral  separation  is  calculated  for  which  two  peaks  rather  than  one  appear. 
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4.2  Verification  of  theoretical  results 

The  format  of  the  experiments  in  this  section  is  now  briefly  described.  The  correlations  used 
were  exact  and  corresponded  to  a  complex  exponential  in  noise  at  The  values  of  SNR  used  ranged 

from  .125  (in  absolute  measurements,  not  dB)  to  16,  in  multiplicative  steps  of  2.  In  the  set  of  experi¬ 
ments  labeled  "a",  the  number  of  correlations  M ,  was  kept  constant  at  40.  The  model  order  was  then 
increased  from  10  to  40  in  additive  steps  of  10.  In  the  set  of  experiments  labeled  "b",  the  model  order 
P  was  kept  constant  at  10.  Then  the  number  of  correlations  used  was  increased  from  10  to  40  in  steps 
of  10.  The  empirical  values  are  denoted  by  circles  connected  by  dashed  lines,  while  the  theoretical 
prediction  are  solid  straight  lines.  Although  the  specific  range  of  values  used  is  somewhat  arbitrary, 
they  do  include  both  a  region  where  the  theoretical  results  are  valid  and  a  region  where  the  empirical 
calculations  deviate  from  the  predicted  values. 

4.2.1  Peak  value  vs.  P,  M,  and  SNR 

In  this  set  of  plots  the  peak  value  for  several  combinations  of  M,  P,  and  SNR  were  calculated. 
In  the  first  plot,  fig.  4.2.1a,  the  number  of  correlations  was  kept  constant  and  the  model  order 
increased.  The  empirical  results  are  compared  with  (3.2.25)  which  indicate  that  there  is  excellent 
agreement  for  SNR  larger  than  1.  In  figure  4.2.1b,  the  model  order  was  kept  constant  and  the 
number  of  correlations  increased.  Again  for  large  SNR,  the  theoretical  predictions  and  the  empirical 
results  agree.  In  these  plots  we  can  see  dearly  the  "SNR  increasing"  property  of  increasing  M .  Note 
that  for  constant  M  SNR,  the  peak  value  remains  the  same.  This  further  verifies  the  theoretical 
results. 

4.2.2  Bandwidth  vs.  P,  M,  and  SNR 

In  this  section,  the  empirical  half-power  bandwidth  is  calculated  and  compared  to  the  theoretical 
predictions  in  (3.2.26).  The  radian  frequency  bandwidth  was  multiplied  by  l/(2tr)  to  convert  the  result 
to  normalized  frequency.  In  figure  4.2.2a  we  find  increasing  agreement  between  the  empirical  and 


-44- 


theoretical  results  as  the  SNR  is  increased  (the  miraculous  agreement  over  all  SNR  for  P-  20  is 
accidental  and  due  to  the  change  in  convexity  of  the  empirical  curve.)  The  second  plot,  figure  4.2.2b, 
shows  the  empirical  calculations  for  die  case  where  the  model  order  is  kept  at  P  =10.  As  M  increases, 
die  asymptotic  behavior  of  the  spectral  bandwidth  approaches  the  predicted  values.  Finally,  if  we  con¬ 
sider  constant  M  SNR  combinations,  we  find  that  the  bandwidth  of  the  spectral  peak  remains  constant 
as  predicted  in  section  3.2. 

4.2.3  Spectral  Area  vs.  P,  M,  SNR 

The  following  plots  study  the  use  of  OSNE  as  a  spectral  density  estimator,  hi  figures  4.2.3a  and 
4.2.3b,  the  spectral  area  is  plotted  as  a  function  of  SNR,  P ,  and  h* .  Unlike  the  ME  case,  the  asymp¬ 
totic  behavior  of  the  spectral  area  depends  an  the  values  of  M  and  P  even  for  large  P.  It  is  not 
surprising  that  good  agreement  is  found  between  empirical  and  theoretical  results  since  both  were  cal¬ 
culated  from  the  results  presented  in  sections  4.2.1  and  4.2.2.  A  more  interesting  result  is  shown  in 
figures  4.2.3c  and  4.2.3d.  Here  each  value  of  the  spectral  area  was  multiplied  by  (P  +  1)(M  +  1)/4M2 
to  get  rid  of  the  dependence  on  the  algorithm  parameters.  We  note  here  that  since  we  are  dealing 
with  a  single  /flcomplex/fR  exponential,  the  formulas  in  table  3.2.2  must  be  modified  accordingly  to 
yield  the  above  proportionality  constant.  We  see  that  for  large  SNR,  multiplying  the  spectral  area  by 
the  above  constant  produces  a  good  estimate  of  the  signal  to  noise  ratio  of  the  sinusoid  power  to  the 
surrounding  noise. 

4.2.4  Interference  among  sinusoids  vs.  spectral  separation 

The  experiments  in  previous  sections  considered  the  case  of  a  single  complex  exponential  in 
noise.  This  section  studies  the  effect  the  interference  of  the  energy  at  negative  frequencies  on  the 
characteristics  of  the  spectral  peak  of  a  single  sinusoid  in  noise.  A  study  of  this  phenomenon  also 
points  out  the  minimum  separation  required  between  two  sinusoids  so  that  the  peak  decoupling  pro¬ 
perty  becomes  valid. 

The  OSNE  method  using  P- 20  and  Af  =40  was  applied  to  a  signal  with  SNR  =  4.  From 
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previous  experiments,  we  see  that  these  values  fall  in  the  region  of  validity  of  the  theoretical  predic¬ 
tions.  The  normalized  frequency,  / ,  of  the  sinusoid  was  changed  from  .01  to  .25  in  steps  of  .01.  The 
values  for  peak  height,  bandwidth,  and  spectral  area  were  converted  to  a  dB  scale  and  normalized  to 
the  value  / =.25.  The  results  of  the  experiments  are  shown  in  figures  4.2.4a,  4.2.4b,  and  4.2.4c. 
From  4.2.4c  we  see  that  the  spectral  area  does  not  vary  by  more  than  .5  dB  until  a  value  of  /  of  about 
.05.  That  means  that  a  minimum  separation  of  .1  is  needed  for  independent  estimation  of  power  for 
the  above  choice  of  P ,  M ,  and  SNR. 

For  comparison,  the  same  experiment  was  conducted  using  P  =20,  M = 20 ,  i.e. ,  the  ME  method, 
figures  4.2.4d,  4.2. 4e,  4.2.41.  In  this  case,  although  the  peak  value  and  bandwidth  vary  tremendously, 
the  spectral  area  remains  very  stable,  varying  by  about  .01  dB  down  to  f-.04.  We  can  conclude  from 
this  that  ME  is  a  more  "stable”  power  estimator;  that  is,  it  is  much  less  sensitive  to  interference  from 
other  sinusoids.  As  a  final  comparison,  the  above  experiment  was  repeated  for  P  =40,  M  =40,  figures 
4.2.4g,  4.2. 4h,  4.2.41.  Here  the  spectral  area  estimate  is  even  more  stable,  further  supporting  the 
assertion  that  ME  is  a  "stable"  power  estimator. 

4.2.5  Resolution  of  two  sinusoids  vs.  P,  M,  SNR 

The  final  experiment  performed  to  study  the  behavior  of  OSNE  was  to  gain  some  idea  of  the 
resolution  capabilities  of  the  method.  It  was  found  that  an  approximate  "rule  of  thumb"  could  be 
extracted  which  described  the  relationship  between  P ,  M ,  and  SNR  and  the  resolving  frequency.  The 
experimental  data  is  plotted  in  figures  4.2.5a  to  figure  4.2.5d  along  with  predictions  based  on  this 
approximate  formula.  Before  discussing  these  results  however,  some  definition  of  resolution  must  be 
presented.  The  definition  used  here  is  in  terms  of  the  convexity  of  the  spectrum  midway  between  the 
location  of  two  equal  amplitude  complex  exponentials.  When  the  separation  between  the  two 
exponentials  is  such  that  the  convexity  of  the  spectrum  at  the  midpoint  is  about  to  change  sign  then 
the  spectrum  is  about  to  change  from  displaying  one  to  two  peaks.  In  this  experiment,  a  real  sinusoid 
near  zero  was  used  and  the  problem  of  resolution  was  rephrased  as  whether  the  resulting  spectrum 
showed  a  peak  at  zero,  implying  that  the  spectral  peaks  at  positive  and  negative  frequencies  were  not 
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figure  4.2.4H  -  bandwidth  versus  sinusoid  loc.,  It“40,  P*40,  SNR"4. 
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flgur*  4.2.51  -  area  versus  sinusoid  loc.,  0*40,  P*40,  SNR“4. 
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separated,  or  whether  the  peak  occurred  at  a  positive  frequency,  so  that  the  two  peaks  at  positive  and 
negative  frequencies  appeared. 


I* 

jr 


! r 


1  r 


Using  the  expression  below  far  the  predictor  Fourier  transform, 


A(«i>)  =  1  +  2e~Jui°k 


the  second  derivative  of  \A  («)  |  is  easily  palmlat^ri  as 


dot2 


=  2 [(Sj)2  -  StS3] 


where 


(4.2.5. 1) 

(4.2.5.2) 


Si  *  i  + 

*»t 

s2  =  2  k  ak 

*-t 

53  =  2k2°k 

i-l 


(4.2.5. 3) 


d2\SM(o))\2  d2|A(<u')l2 

It  is  readily  seen  that  — 1  JL-  is  monotonic  with - 1  v1  and  is  equal  to  zero  at  the 

dm2  dot2 

same  values  of  u>.  Thus  we  can  use  (4.2.5.2)  and  (4.2.5.3)  to  decide  whether  a  sinusoid  located  at  w0 
win  be  resolved  for  a  given  value  of  P ,  M ,  and  SNR.  The  procedure  is  to  see  if  (4,2.5.2)  is  less  than 


or  greater  than  zero.  If  greater  than  zero,  then  the  sinusoid  is  certainly  resolved;  if  less  than  zero, 
there  is  the  possibility  that  the  sinusoid  is  still  resolved  if  u0  is  much  greater  than  zero.  What  could 
occur  is  that  the  oscillations  of  the  spectrum  about  0  dB  as  seen  in  figure  3.2.1  may  yield  a  negative 
convexity  at  zero.  However,  if  we  are  careful  not  to  wander  too  far  from  <i>0-0,  checking  if  (4. 2.5. 2) 
is  less  than  or  greater  than  zero  win  provide  an  accurate  test  of  resolvability. 


Far  combinations  of  P,  M,  and  SNR,  the  sinusoid  location  was  varied  until  the  the  value  of 
(4.2.5.2)  equaled  0.  In  figure  4.2.5a,  M  was  kept  constant  at  40,  and  P  and  the  SNR  varied.  In  fig¬ 
ure  4.2.5b,  P  was  now  kept  constant  at  20  and  M  and  the  SNR  varied.  From  the  linear  characteristic 
of  the  resolution  as  a  function  of  SNR  we  can  deduce  that  the  resolving  frequency,  for  a  constant  P 
and  M  varies  as  a  function  of  some  power  of  the  SNR  times  a  constant  which  depends  on  P  and  M 
only.  In  figures  4.2.5c  and  4.2.5d,  the  SNR  was  kept  constant  at  16.  and  the  resolving  frequency  plot- 
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ted  as  a  function  of  M  and  P .  In  all  the  plots,  we  see  approximate  linear  relationships  (on  a  log  scale) 
between  M ,  P ,  SNR,  and  the  resolving  frequency  as  the  SNR,  P ,  and  Af  increase.  Such  a  behavior 
implies  that  the  relation  between  the  resolving  frequency  and  the  above  parameters  is  given  by, 

frud  55  kAt'"Pa'SNiram  (4.2.5. 4) 

From  the  experimental  data  we  can  extract  the  above  parameters  (the  data  points  used  were  the  com¬ 
binations  (P,  M,  SNR):  (40,  40,  16),  (40,  40,  g),  (30,  40,  16),  (30,  30,  16)  ) 

*  -  .47 
a,  -  -.67 
aM  =  -.63 
aSM»  =  “-32 

Using  the  above  model  the  predicted  results  of  the  experiments  are  plotted  in  the  solid  lines  in  figures 
4.2.5a,  4.2.5b,  4.2.5c,  4.2.5d.  The  reader  should  not  be  surprised  at  the  agreement  between  the 
predicted  values  and  the  experimental  values  for  the  combinations  described  in  the  previous  paragraph; 
after  all,  it  was  using  that  data  that  the  model  was  derived.  However,  there  is  some  widespread  agree¬ 
ment  in  other  regions  of  the  figures.  These  examples  seem  to  point  to  the  structure  in  (4.2. 5. 4)  as 
being  of  the  right  form.  However,  no  more  decisive  comments  than  that  will  be  made.  A  theoretical 
analysis  of  the  resolution  properties  of  OSNE  does  not  seem  intractable  using  the  techniques  described 
in  this  thesis  and  may  provide  some  verification  of  the  empirical  results  presented  here. 


CHAPTER  5 

SUMMARY  AND  CONCLUSIONS 


la  this  thesis  a  study  is  presented  of  some  of  the  properties  of  the  method  of  overspecified  nor¬ 
mal  equations  as  applied  to  the  problem  of  spectral  estimation.  The  main  contribution  of  this  thesis  is 
the  derivation  of  the  relationships  between  the  number  of  correlations  used,  the  model  order  and  the 
signal  to  noise  ratio  of  the  signal,  to  the  characteristics  of  the  resulting  spectral  estimate  in  terms  of  the 
spectral  height,  bandwidth  and  area.  The  method  is  shown  to  be  a  spectral  density  estimator  like  the 
ME  method,  where  spectral  areas  rather  than  spectral  values  should  be  interpreted  as  estimates  of 
power. 

The  relationships  derived  point  to  the  number  of  correlations  used  over  the  minimum,  i.e.  model 
order,  as  a  signal-to-noise  enhancer,  where  the  resulting  spectrum  is  equivalent  to  the  ME  spectrum 
under  higher  signal-to-noise  conditions.  Another  result  is  the  requirement  of  a  proportionality  con¬ 
stant  dependent  on  the  number  of  correlations  and  the  model  order  which  is  required  for  accurate 
signal-to-noise  measurements.  This  constant  is  not  required  however,  for  measurements  of  reiative 
power  within  the  same  spectral  estimate,  as  in  the  power  ratio  of  two  sinusoids  in  noise. 

The  second  part  of  the  thesis  presents  some  empirical  studies  using  computer  simulations  which 
verify  the  theoretical  predictions  and  provide  the  region  of  validity  of  the  analysis.  Further  experi¬ 
ments  study  the  interfering  effect  of  several  closely  spaced  sinusoids.  The  method  of  overspecified  nor¬ 
mal  equations  is  shown  to  be  much  more  sensitive  to  this  interference  than  the  ME  method.  Finally, 
some  purely  empirical  studies  are  made  of  the  resolution  capabilities  of  the  method.  Using  the  data 
derived,  an  empirical  model  is  derived  which  seems  to  agree  to  some  extent  with  the  data. 
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